Skip to content

docs: rewrite README with "Why spec-forge?" section and fix unclosed code fence#17

Merged
spencercjh merged 8 commits intomainfrom
copilot/update-readme-for-spec-forge
Mar 8, 2026
Merged

docs: rewrite README with "Why spec-forge?" section and fix unclosed code fence#17
spencercjh merged 8 commits intomainfrom
copilot/update-readme-for-spec-forge

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 8, 2026

The README read like an API reference with no explanation of why spec-forge exists — the real pain points (abandoned gRPC tooling, swaggo annotation burden, go-zero bugs) were invisible to new visitors.

Changes

New "Why spec-forge?" section

  • gRPC/Protobuf: calls out google/gnostic (abandoned), grpc-gateway (Swagger 2.0 only), and buf (no official OpenAPI docs) — explains why protoc-gen-connect-openapi was chosen
  • Gin: shows the swaggo/swag annotation problem (compiler-unvalidated, manual sync) vs. spec-forge's zero-annotation AST approach
  • Hertz/Kitex: notes tooling exists but is officially undocumented
  • go-zero: clarifies that goctl swagger bugs are patched internally
  • vs AI-generated docs: distinguishes code-synchronized enrichment from LLM-written docs that drift
  • AI Agent angle: positions accurate OpenAPI specs as machine-readable interface contracts

Updated one-liner

A CLI tool that solves the fragmented, painful world of OpenAPI spec generation — auto-detects your framework, generates accurate specs from source code, and enriches them with AI.

Structural improvements

  • Added "Zero annotations for Gin" to the Features list
  • Moved Supported Frameworks table before Framework-Specific Usage
  • Added inline callout blockquotes to the Gin and gRPC sections reinforcing the key differentiators
  • Expanded Quick Start to cover all four supported frameworks

Bug fix

  • Closed the missing ``` fence in the ### Publishing to ReadMe.com section that left the code block unclosed before ## License
Original prompt

Background

The current README is functional but too neutral — it reads like an API reference rather than a compelling pitch. Many developers question the value of spec-forge, but the project actually solves very real, concrete pain points that are not communicated anywhere in the current README.

After a deep discussion about the project's positioning, the following key insights emerged that should be reflected in the README:

Core Pain Points spec-forge Solves (that are NOT in the current README)

1. gRPC/Protobuf ecosystem is fragmented and poorly documented:

  • google/gnostic's protoc-gen-openapi is abandoned/unmaintained — yet still referenced by most tutorials
  • grpc-gateway's protoc-gen-openapiv2 only outputs Swagger 2.0, not OpenAPI 3.x
  • buf has no official documentation for OpenAPI generation — developers rely entirely on third-party blogs
  • Developers waste hours figuring out which tool to use before writing a single line of code

2. Gin requires verbose godoc annotations (swaggo pain):

  • The dominant solution (swaggo/swag) requires writing // @Summary, // @Param, // @Success etc. for every handler
  • These annotations are not validated by the Go compiler — errors only surface at generation time
  • Renaming a type means manually updating annotations everywhere
  • spec-forge's AST-based approach requires zero annotations — this is a key differentiator that is completely missing from the README

3. Hertz and Kitex (CloudWeGo ecosystem) have hidden, undocumented tooling:

  • OpenAPI generation tools exist but are buried in GitHub corners with no official documentation
  • The official CloudWeGo docs don't mention how to generate OpenAPI specs
  • spec-forge is the only tool that wraps this complexity with a single command

4. go-zero's goctl swagger has known bugs:

  • spec-forge patches these bugs internally (this is real implementation work that's invisible to users)

5. AI-enriched specs are fundamentally different from "AI-generated" specs:

  • A common objection is "just use ChatGPT to write your docs"
  • The README must clearly explain: AI generation ≠ code-synchronized spec
  • spec-forge reads the actual code structure, then uses AI to fill in human-readable descriptions — the output is both machine-accurate and human-readable

6. The AI Agent era makes high-quality OpenAPI specs increasingly critical:

  • OpenAPI specs are becoming the "interface contract" for AI agents to call APIs
  • This is a new, compelling reason to maintain accurate specs that the README doesn't mention at all

What Needs to Change in README.md

Please rewrite the README.md with the following structure. Preserve all existing technical content (configuration, commands, framework-specific usage, etc.) — only add and reorganize, don't remove working documentation.

New README Structure:

  1. Title + Badges (keep as-is)

  2. One-liner — upgrade from the current neutral description to something that leads with the problem:

    A CLI tool that solves the fragmented, painful world of OpenAPI spec generation — auto-detects your framework, generates accurate specs from source code, and enriches them with AI.

  3. NEW: "Why spec-forge?" section — this is the most important addition. Must include:

    • A clear statement that generating OpenAPI from backend code is harder than it should be, with a concrete breakdown by ecosystem:

      • gRPC/Protobuf: google/gnostic is abandoned, grpc-gateway only outputs Swagger 2.0, buf has no official OpenAPI docs (only community blogs), leaving developers lost
      • Gin: swaggo requires hundreds of verbose // @swagger annotations that the compiler never validates — spec-forge uses AST analysis requiring zero annotations
      • Hertz / Kitex (CloudWeGo): tools exist but are buried with no official documentation
      • go-zero: goctl swagger has known bugs that spec-forge patches internally
    • A "vs AI-generated docs" explanation:

      spec-forge reads your actual code structure to guarantee accuracy, then uses AI to make it human-readable. This is fundamentally different from asking an LLM to write docs from scratch — those drift from reality the moment your code changes.

    • A mention of the AI Agent angle:

      In the AI Agent era, OpenAPI specs are becoming machine-readable interface contracts. spec-forge helps ensure yours are accurate and complete.

  4. Features (keep existing 4 bullets, consider adding one about "zero annotation" for Gin)

  5. How It Works (keep existing mermaid diagram + step descriptions)

  6. Installation (keep as-is)

  7. Quick Start (keep as-is, but make the examples span more frameworks — not just Spring Boot)

  8. Supported Frameworks (keep existing table)

  9. Framework-Specific Usage — keep all existing content, but add a clear callout for Gin explaining the "no annotations needed" benefit at the top of...

This pull request was created from Copilot chat.


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

…ence fix

Co-authored-by: spencercjh <29922079+spencercjh@users.noreply.github.com>
Copilot AI changed the title [WIP] Update README to highlight value of spec-forge docs: rewrite README with "Why spec-forge?" section and fix unclosed code fence Mar 8, 2026
@spencercjh spencercjh requested review from Copilot and spencercjh March 8, 2026 13:34
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR rewrites the README to add a "Why spec-forge?" section that explains the concrete pain points solved by the tool (fragmented gRPC tooling, swaggo annotation burden for Gin, undocumented CloudWeGo tools, go-zero goctl bugs), updates the one-liner, reorganizes existing content for better flow, and fixes an unclosed code fence bug.

Changes:

  • Added a new "Why spec-forge?" section with per-framework pain-point explanations, an LLM comparison, and an AI Agent angle
  • Reorganized sections (Supported Frameworks table moved before Framework-Specific Usage, Configuration/LLM Providers moved after framework details), expanded Quick Start with examples for all four frameworks, and added callout blockquotes in Gin and gRPC sections
  • Fixed a missing closing ``` fence in the "Publishing to ReadMe.com" section

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread README.md Outdated
spencercjh and others added 2 commits March 8, 2026 21:55
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
- Change google/gnostic status from 'abandoned' to 'inactive'
- Update Hertz/Kitex docs with hertz-contrib/swagger-generate references
- Remove unconfirmed go-zero bug mentions
- Split framework details into separate docs to reduce README size
- Add docs/spring-boot.md, gin.md, go-zero.md, grpc-protoc.md, hertz.md, kitex.md

Signed-off-by: spencercjh <spencercjh@gmail.com>
@spencercjh spencercjh marked this pull request as ready for review March 8, 2026 14:53
Copilot AI review requested due to automatic review settings March 8, 2026 14:53
@qodo-code-review
Copy link
Copy Markdown

Review Summary by Qodo

Rewrite README with framework pain points and extract framework docs

📝 Documentation ✨ Enhancement

Grey Divider

Walkthroughs

Description
• Rewrote README with "Why spec-forge?" section explaining pain points in OpenAPI generation across
  frameworks
• Added comprehensive framework-specific documentation files (Gin, gRPC, go-zero, Spring Boot,
  Hertz, Kitex)
• Moved framework details to separate docs to reduce README size and improve maintainability
• Enhanced Quick Start section with examples for all four supported frameworks
• Added "Zero annotations for Gin" feature highlight and improved positioning against competing
  tools
• Fixed unclosed code fence in Publishing to ReadMe.com section
Diagram
flowchart LR
  A["README.md<br/>Original"] -->|"Add Why section<br/>& framework callouts"| B["README.md<br/>Refactored"]
  A -->|"Extract framework<br/>details"| C["docs/gin.md"]
  A -->|"Extract framework<br/>details"| D["docs/grpc-protoc.md"]
  A -->|"Extract framework<br/>details"| E["docs/go-zero.md"]
  A -->|"Extract framework<br/>details"| F["docs/spring-boot.md"]
  A -->|"Extract framework<br/>details"| G["docs/hertz.md"]
  A -->|"Extract framework<br/>details"| H["docs/kitex.md"]
  B -->|"Link to"| C
  B -->|"Link to"| D
  B -->|"Link to"| E
  B -->|"Link to"| F
Loading

Grey Divider

File Changes

1. README.md 📝 Documentation +86/-67

Rewrite README with value proposition and framework pain points

• Replaced generic one-liner with compelling value proposition addressing fragmented OpenAPI tooling
• Added "Why spec-forge?" section with detailed pain points for gRPC, Gin, Hertz/Kitex, and go-zero
 ecosystems
• Added comparison section explaining AI-enriched specs vs AI-generated docs and AI Agent angle
• Expanded Quick Start with examples for Gin, gRPC, go-zero, and AI enrichment
• Moved Supported Frameworks table before Framework-Specific Usage section
• Added "Zero annotations for Gin" to Features list
• Removed verbose framework-specific usage sections (moved to separate docs)
• Fixed unclosed code fence in Publishing to ReadMe.com section

README.md


2. docs/gin.md 📝 Documentation +55/-0

New Gin framework documentation with zero-annotation approach

• New file explaining Gin framework support with AST-based static analysis
• Detailed comparison with swaggo/swag annotation burden
• Documents supported patterns (route registration, groups, middleware, parameter binding)
• Provides usage examples with and without AI enrichment

docs/gin.md


3. docs/go-zero.md 📝 Documentation +32/-0

New go-zero framework documentation and usage guide

• New file documenting go-zero project support
• Explains detection, patching, and generation workflow
• Lists prerequisites (goctl tool installation)
• Provides basic and enriched generation examples

docs/go-zero.md


View more (4)
4. docs/grpc-protoc.md 📝 Documentation +51/-0

New gRPC/Protobuf documentation with tooling comparison

• New file explaining gRPC/Protobuf support using native protoc
• Includes comparison table of gRPC tooling landscape (gnostic, grpc-gateway, buf)
• Documents protoc-gen-connect-openapi as the chosen solution
• Lists prerequisites and installation steps
• Provides usage examples with import paths and AI enrichment
• Notes buf-managed projects limitation

docs/grpc-protoc.md


5. docs/spring-boot.md 📝 Documentation +34/-0

New Spring Boot framework documentation and setup guide

• New file documenting Spring Boot project support
• Explains springdoc-openapi integration and detection workflow
• Documents supported build tools (Maven and Gradle)
• Covers multi-module Maven project handling
• Provides usage examples with and without AI enrichment

docs/spring-boot.md


6. docs/hertz.md 📝 Documentation +22/-0

New Hertz framework documentation with coming soon status

• New file for Hertz (CloudWeGo) framework documentation
• Marks feature as coming soon with 🚧 status
• References hertz-contrib/swagger-generate and protoc-gen-http-swagger as recommended tools
• Provides links to official repositories

docs/hertz.md


7. docs/kitex.md 📝 Documentation +22/-0

New Kitex framework documentation with coming soon status

• New file for Kitex (CloudWeGo) framework documentation
• Marks feature as coming soon with 🚧 status
• References hertz-contrib/swagger-generate and protoc-gen-rpc-swagger as recommended tools
• Provides links to official repositories

docs/kitex.md


Grey Divider

Qodo Logo

@qodo-code-review
Copy link
Copy Markdown

qodo-code-review Bot commented Mar 8, 2026

Code Review by Qodo

🐞 Bugs (2) 📘 Rule violations (1) 📎 Requirement gaps (0)

Grey Divider


Action required

1. Wrong OpenAI env var🐞 Bug ✓ Correctness
Description
README examples use LLM_API_KEY with --provider openai, but the CLI requires OPENAI_API_KEY;
users following Quick Start will hit immediate auth failures.
Code

README.md[R109-115]

+# Generate with AI enrichment (any framework)
+LLM_API_KEY="your-api-key" spec-forge generate ./path/to/project \
  --enrich --provider openai --model gpt-4o --language en
# Enrich an existing OpenAPI spec
LLM_API_KEY="your-api-key" spec-forge enrich ./openapi.json \
  --provider openai --model gpt-4o --language zh
Evidence
README’s Quick Start shows LLM_API_KEY while selecting the OpenAI provider, but the implementation
explicitly reads OPENAI_API_KEY for provider=openai and errors if it is not set; README’s own
provider table also states OpenAI uses OPENAI_API_KEY.

README.md[109-116]
README.md[161-168]
cmd/enrich.go[188-205]
cmd/generate.go[357-371]
internal/enricher/config.go[80-96]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
README shows `LLM_API_KEY` while explicitly selecting `--provider openai`, but the CLI requires `OPENAI_API_KEY` for OpenAI. This will cause immediate authentication failures for users following Quick Start.
### Issue Context
The CLI hard-codes provider-specific env vars for OpenAI/Anthropic in both `generate` enrichment and `enrich` command paths.
### Fix Focus Areas
- README.md[109-116]
- README.md[161-168]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

2. Spring docs omit wrappers 📘 Rule violation ⛯ Reliability
Description
The new Spring Boot documentation states the tool runs mvn verify / gradle generateOpenApiDocs
without mentioning wrapper-first execution (./mvnw / ./gradlew). This conflicts with the
required wrapper-preference behavior and may encourage less reproducible builds.
Code

docs/spring-boot.md[R13-16]

+| Build Tool | Command Used |
+|------------|--------------|
+| Maven | `mvn verify` |
+| Gradle | `gradle generateOpenApiDocs` |
Evidence
Compliance requires Maven/Gradle execution to prefer project wrappers before falling back to system
binaries. The added Spring Boot docs explicitly list system mvn/gradle commands as the commands
used, with no wrapper-first guidance.

CLAUDE.md
docs/spring-boot.md[13-16]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The Spring Boot documentation lists `mvn verify` and `gradle generateOpenApiDocs` as the commands used, without indicating wrapper-first execution. This conflicts with the compliance requirement to prefer `./mvnw` / `./gradlew` before system binaries.
## Issue Context
Compliance requires Maven/Gradle execution to prefer project wrappers (including multi-module parent wrappers) before falling back to system `mvn`/`gradle`. The new docs should reflect this behavior (or at least not contradict it).
## Fix Focus Areas
- docs/spring-boot.md[13-16]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. go-zero command mismatch🐞 Bug ✓ Correctness
Description
New go-zero docs (and README narrative) describe the wrong goctl prerequisite/command
(goctl-swagger / goctl swagger), but the implementation checks for goctl and runs `goctl api
swagger`; docs will mislead users during setup/troubleshooting.
Code

docs/go-zero.md[R7-9]

+1. **Detection**: Parses `go.mod` to detect go-zero dependency and locate API definition files (`.api` files)
+2. **Patching**: Checks for `goctl-swagger` installation
+3. **Generation**: Uses `goctl swagger` command to generate the OpenAPI spec
Evidence
The newly added go-zero docs instruct users about goctl-swagger and goctl swagger, but the
go-zero extractor code verifies goctl --version and executes goctl with args api swagger.
README’s new 'Why spec-forge?' section repeats the incorrect goctl swagger phrasing.

docs/go-zero.md[7-10]
README.md[45-47]
internal/extractor/gozero/patcher.go[40-62]
internal/extractor/gozero/generator.go[189-218]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Documentation for go-zero currently describes incorrect tooling/command names (`goctl-swagger` and `goctl swagger`). The implementation uses `goctl` and runs `goctl api swagger`.
### Issue Context
Users relying on docs may install the wrong tool or run the wrong command while debugging failures.
### Fix Focus Areas
- docs/go-zero.md[7-9]
- README.md[45-47]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

ⓘ The new review experience is currently in Beta. Learn more

Grey Divider

Qodo Logo

Comment thread README.md Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/go-zero.md Outdated
Comment thread docs/spring-boot.md Outdated
Comment thread docs/gin.md Outdated
Comment thread docs/grpc-protoc.md Outdated
Comment thread docs/go-zero.md Outdated
Comment thread README.md Outdated
- Fix incorrect 'goctl-swagger' to 'goctl'
- Fix incorrect 'goctl swagger' to 'goctl api swagger' to match implementation

Signed-off-by: spencercjh <spencercjh@gmail.com>
Clarify that spec-forge prefers ./mvnw / ./gradlew over system binaries,
including support for parent directory wrappers in multi-module projects.

Signed-off-by: spencercjh <spencercjh@gmail.com>
Use OPENAI_API_KEY instead of LLM_API_KEY when --provider openai is specified,
to match the CLI's provider-specific environment variable requirements.

Signed-off-by: spencercjh <spencercjh@gmail.com>
Remove invalid --enrich, --provider, and --model flags from generate
command examples. The generate command only supports --language for
enrichment configuration; enrichment is triggered via .spec-forge.yaml
or automatically when configured. --provider and --model only exist
on the enrich subcommand.

Signed-off-by: spencercjh <spencercjh@gmail.com>
Copy link
Copy Markdown
Owner

@spencercjh spencercjh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@spencercjh spencercjh merged commit b368ea7 into main Mar 8, 2026
4 checks passed
@spencercjh spencercjh deleted the copilot/update-readme-for-spec-forge branch March 8, 2026 15:14
spencercjh added a commit that referenced this pull request Mar 8, 2026
…code fence (#17)

* Initial plan

* docs: rewrite README with Why section, framework callouts, and code fence fix

Co-authored-by: spencercjh <29922079+spencercjh@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* docs: refine README and add framework-specific documentation

- Change google/gnostic status from 'abandoned' to 'inactive'
- Update Hertz/Kitex docs with hertz-contrib/swagger-generate references
- Remove unconfirmed go-zero bug mentions
- Split framework details into separate docs to reduce README size
- Add docs/spring-boot.md, gin.md, go-zero.md, grpc-protoc.md, hertz.md, kitex.md

Signed-off-by: spencercjh <spencercjh@gmail.com>

* docs: fix go-zero tooling/command names

- Fix incorrect 'goctl-swagger' to 'goctl'
- Fix incorrect 'goctl swagger' to 'goctl api swagger' to match implementation

Signed-off-by: spencercjh <spencercjh@gmail.com>

* docs: add wrapper priority note to spring-boot.md

Clarify that spec-forge prefers ./mvnw / ./gradlew over system binaries,
including support for parent directory wrappers in multi-module projects.

Signed-off-by: spencercjh <spencercjh@gmail.com>

* docs: fix OpenAI API key environment variable in examples

Use OPENAI_API_KEY instead of LLM_API_KEY when --provider openai is specified,
to match the CLI's provider-specific environment variable requirements.

Signed-off-by: spencercjh <spencercjh@gmail.com>

* docs: fix generate command enrichment examples

Remove invalid --enrich, --provider, and --model flags from generate
command examples. The generate command only supports --language for
enrichment configuration; enrichment is triggered via .spec-forge.yaml
or automatically when configured. --provider and --model only exist
on the enrich subcommand.

Signed-off-by: spencercjh <spencercjh@gmail.com>

---------

Signed-off-by: spencercjh <spencercjh@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: spencercjh <29922079+spencercjh@users.noreply.github.com>
Co-authored-by: Spencer Cai <spencercjh@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
spencercjh added a commit that referenced this pull request Mar 30, 2026
Review feedback addressed:
- CustomProvider: use p.name instead of hardcoded CustomProviderName in error messages
- StreamWriter: optimize bytes.Contains to bytes.IndexByte for hot path
- E2E tests: handle enrich.enabled=false config, pass --custom-api-key-env for custom provider
- Design doc: fix StreamWriter file path, update Stream to *bool type, correct prefix casing

Fixes review comments #11, #12, #13, #16, #17 on PR #57

Signed-off-by: caijiahao <caijh@inesa.com>
Signed-off-by: spencercjh <spencercjh@gmail.com>
spencercjh added a commit that referenced this pull request Mar 30, 2026
* docs: add Phase 4.1 Streaming design document

Design P4.1: Real-time streaming support for LLM enrichment

Key features:
- Option pattern for Provider interface (WithStreamingFunc)
- StreamWriter for thread-safe output with batch prefix
- Concurrent processing with mutex-protected output
- CLI flag --no-stream to disable streaming (enabled by default)

Signed-off-by: spencercjh <spencercjh@gmail.com>

* docs: add P4.1 Streaming implementation plan

8 tasks with TDD approach:
1. Provider interface refactoring (Option pattern)
2. StreamWriter implementation
3. Provider streaming implementation
4. BatchProcessor streaming integration
5. Enricher streaming support
6. CLI integration (--no-stream flag)
7. Integration testing
8. Final verification

Signed-off-by: spencercjh <spencercjh@gmail.com>

* feat(provider): add Option pattern for streaming support

- Add GenerateOptions, Option, WithStreamingFunc, applyOptions
- Update Provider interface to accept ...Option
- Update all provider implementations with new signature (streaming logic in next commit)

Signed-off-by: spencercjh <spencercjh@gmail.com>

* feat(processor): add StreamWriter for thread-safe streaming output

- StreamWriter with mutex-protected writes
- WriteWithPrefix adds batch type prefix
- Tested for concurrent access safety

Signed-off-by: spencercjh <spencercjh@gmail.com>

* feat(provider): add streaming support to all providers

- Replace GenerateFromSinglePrompt with GenerateContent
- Pass StreamingFunc to langchaingo via llms.WithStreamingFunc
- Return content from response.Choices for non-streaming callers

Signed-off-by: spencercjh <spencercjh@gmail.com>

* feat(processor): integrate streaming into BatchProcessor

- Add StreamWriter field and WithStreamWriter option
- Pass streaming callback to provider with batch type prefix
- Update NewBatchProcessor to accept functional options

Task 4 of P4.1 streaming implementation

Signed-off-by: spencercjh <spencercjh@gmail.com>

* feat(enricher): add Stream option to EnrichOptions

- Add Stream (bool) and Writer (io.Writer) to EnrichOptions
- Create StreamWriter when streaming is enabled
- Default: streaming enabled to os.Stdout

Signed-off-by: spencercjh <spencercjh@gmail.com>

* feat(cli): add --no-stream flag to enrich command

- Streaming enabled by default
- Use --no-stream to disable and wait for complete response

Signed-off-by: spencercjh <spencercjh@gmail.com>

* test(enricher): add streaming integration test

- Test that streaming callback is invoked
- Test that StreamWriter output contains batch prefix [api]

Signed-off-by: spencercjh <spencercjh@gmail.com>

* style(enricher): fix whitespace alignment

Signed-off-by: spencercjh <spencercjh@gmail.com>

* fix(batch): update streaming function to ignore context parameter

Signed-off-by: spencercjh <spencercjh@gmail.com>

* fix(processor): add nil check to NewStreamWriter

Prevent runtime panic by validating writer parameter is not nil.
Add test to verify panic behavior with nil writer.

Signed-off-by: spencercjh <spencercjh@gmail.com>

* test(enricher): add test for Stream: false path

Add TestEnricher_WithStreamingDisabled to verify that no streaming
callback is passed to the provider when Stream option is disabled.

Signed-off-by: spencercjh <spencercjh@gmail.com>

* test(processor): add streaming callback verification in batch tests

Add TestBatchProcessor_ProcessBatch_WithStreaming to verify that
streaming callback is properly invoked when StreamWriter is configured.

Signed-off-by: spencercjh <spencercjh@gmail.com>

* feat(processor): add chunk buffering to StreamWriter

- Add buffer accumulation for streaming chunks
- Auto-flush on newline, buffer threshold, or prefix change
- Add WithFlushThreshold option for configurable buffering
- Add Flush method for explicit buffer clearing
- Update enricher to flush StreamWriter after processing
- Add tests for buffering behavior

Signed-off-by: spencercjh <spencercjh@gmail.com>

* feat(processor): add streaming debug logging and metrics

- Add StreamWriterMetrics struct to track streaming statistics
- Add GetMetrics() method for retrieving metrics
- Add WithDebug option for enabling debug logging
- Add WithFlushThreshold option for configurable buffering
- Track total chunks, bytes, flushes, and unique prefixes
- Debug logging shows chunk details and flush events

Signed-off-by: spencercjh <spencercjh@gmail.com>

* style: fix formatting issues from golangci-lint

Signed-off-by: spencercjh <spencercjh@gmail.com>

* docs: clarify make verify usage in CLAUDE.md

Explain that make verify checks for uncommitted changes and should only
be used after committing. Recommend individual commands (make fmt, lint,
test) before committing to avoid false failures from git diff check.

Signed-off-by: spencercjh <spencercjh@gmail.com>

* test(e2e): add conditional enrich E2E tests with streaming verification

- Add skipIfNoConfig helper to skip tests without valid E2E config
- Check for .spec-forge.e2e.local.yaml or .spec-forge.local.yaml
- Verify API key environment variable is set before running
- Test streaming output with prefix verification ([api], [schema], [param])
- Test --no-stream flag disables streaming prefixes
- Test local config file loading mechanism
- Add .spec-forge.e2e.example.yaml as configuration template
- Update .gitignore to exclude E2E local configs

Signed-off-by: spencercjh <spencercjh@gmail.com>

* test(e2e): add conditional enrich E2E tests with streaming verification

- Add loadE2EConfig helper to load config from integration-tests/.spec-forge.e2e.local.yaml
- Skip tests gracefully if no valid config found (file missing or API key env not set)
- Add TestE2E_Enrich_NoStreamFlag to verify --no-stream disables streaming prefixes
- Add TestE2E_Enrich_WithStreaming to test real LLM enrichment with streaming output
- Add TestE2E_Enrich_WithLocalConfig to test local config file loading
- Add .spec-forge.e2e.example.yaml as configuration template
- Update .gitignore to exclude integration-tests/.spec-forge.e2e.local.yaml
- Document E2E enrich test setup in integration-tests/README.md

Signed-off-by: spencercjh <spencercjh@gmail.com>

* test(e2e): fix enrich E2E tests for streaming verification

- Remove streaming prefix check in stdout (StreamWriter writes to os.Stdout directly, not Cobra's buffer)
- Remove TestE2E_Enrich_WithLocalConfig (tests Viper config loading, not enrich logic)
- Verify enrichment by checking the output spec file contains descriptions
- Update README to document only the two streaming-related tests
- All tests pass:
  - TestE2E_Enrich_Help: PASS
  - TestE2E_Enrich_MissingArgs: PASS
  - TestE2E_Enrich_NonExistentFile: PASS
  - TestE2E_Enrich_NoStreamFlag: SKIP (requires config)
  - TestE2E_Enrich_WithStreaming: PASS (with config) / SKIP (without config)

Signed-off-by: spencercjh <spencercjh@gmail.com>

* fix(provider): satisfy goconst in anthropic and custom providers

Agent-Logs-Url: https://github.com/spencercjh/spec-forge/sessions/be20b0d1-0dc2-45c4-adc4-b2bec21a9a95

Co-authored-by: spencercjh <29922079+spencercjh@users.noreply.github.com>

* fix(review): address PR review feedback for P4.1 streaming

- Change EnrichOptions.Stream to *bool (tri-state) to avoid backwards
  compatibility issues with zero value of bool (false). Now nil means
  use default (true), explicit false/true overrides.
- Add error return when LLM providers return empty choices to prevent
  silent enrichment failures (OpenAI, Anthropic, Ollama, Custom).
- Fix comment in batch.go to reflect lowercase prefix values.
- Remove stale TestE2E_Enrich_WithLocalConfig entry from README table.
- Make E2E test assertion language-agnostic (check for non-empty
  descriptions instead of specific Chinese text).

Signed-off-by: spencercjh <spencercjh@gmail.com>

* fix(review): address additional PR review feedback for P4.1 streaming

Review feedback addressed:
- CustomProvider: use p.name instead of hardcoded CustomProviderName in error messages
- StreamWriter: optimize bytes.Contains to bytes.IndexByte for hot path
- E2E tests: handle enrich.enabled=false config, pass --custom-api-key-env for custom provider
- Design doc: fix StreamWriter file path, update Stream to *bool type, correct prefix casing

Fixes review comments #11, #12, #13, #16, #17 on PR #57

Signed-off-by: caijiahao <caijh@inesa.com>
Signed-off-by: spencercjh <spencercjh@gmail.com>

* fix(review): address third round of Copilot review feedback for P4.1

Review feedback addressed:
- factory.go: use AnthropicProviderName and CustomProviderName constants instead of string literals
- design doc: update EnrichOptions summary to reflect actual API (Stream *bool, Writer io.Writer)
- design doc: fix data flow example to use lowercase prefix [api] instead of [API]
- stream_writer.go: validate WithFlushThreshold to use DefaultFlushThreshold for zero/negative values
- e2e_enrich_test.go: pass --timeout flag when configured in E2E config

Fixes review comments #21-26 on PR #57

Signed-off-by: caijiahao <caijh@inesa.com>
Signed-off-by: spencercjh <spencercjh@gmail.com>

* fix(review): address fourth round of Copilot review feedback

Review feedback addressed:
- e2e_enrich_test.go: change Enabled to *bool to distinguish between
  "not set" (nil, run tests) and "explicitly false" (skip tests)
- README.md: update skip message path to match actual output
- batch.go: add Flush() call after each LLM call to ensure buffered
  streaming output is visible for short responses

Note: #30 (streaming to os.Stdout) is intentional design for real-time
terminal feedback and not changed.

Fixes review comments #27-29 on PR #57

Signed-off-by: caijiahao <caijh@inesa.com>
Signed-off-by: spencercjh <spencercjh@gmail.com>

* fix(review): address fifth round of Copilot review feedback

Review feedback addressed:
- e2e_enrich_test.go: match API key env var logic to CLI behavior
  (OPENAI_API_KEY, ANTHROPIC_API_KEY, or LLM_API_KEY for custom)
- e2e_enrich_test.go: use YAML parsing for provider-agnostic assertions
  instead of language-specific substring matching
- design doc: update output example to show actual raw JSON streaming
  behavior instead of hypothetical human-readable text

Note: #32 (streaming writes to os.Stdout) is intentional design for
real-time terminal feedback - already addressed in #30 response.

Fixes review comments #31, #33, #34 on PR #57

Signed-off-by: caijiahao <caijh@inesa.com>
Signed-off-by: spencercjh <spencercjh@gmail.com>

* fix(enricher,stream_writer): improve error handling and metrics updates

Signed-off-by: spencercjh <spencercjh@gmail.com>

* feat(batch): add streaming support and improve processing logic

Signed-off-by: spencercjh <spencercjh@gmail.com>

* fix(review): address Copilot review feedback - streaming, concurrency, docs

- Remove redundant atomic ops in StreamWriter, use mutex-only sync
- Handle short writes in flushLocked()
- Change final Flush error to warning (streaming is ancillary)
- Streaming mode processes batches sequentially for readable output
- --no-stream enables concurrent processing (--concurrency applies)
- Update flag descriptions, config comments, design doc, CLAUDE.md

Signed-off-by: spencercjh <spencercjh@gmail.com>

---------

Signed-off-by: spencercjh <spencercjh@gmail.com>
Signed-off-by: caijiahao <caijh@inesa.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: spencercjh <29922079+spencercjh@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants